Skip to content

fix(sqlserver): make EnsureDatabaseExistsAsync safe under concurrent callers (weasel#415) - #418

Merged
jeremydmiller merged 1 commit into
masterfrom
fix/415-ensure-database-exists-race
Jul 31, 2026
Merged

fix(sqlserver): make EnsureDatabaseExistsAsync safe under concurrent callers (weasel#415)#418
jeremydmiller merged 1 commit into
masterfrom
fix/415-ensure-database-exists-race

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #415.

SqlServerMigrator.EnsureDatabaseExistsAsync did a check-then-create against master. Two gaps show up as soon as more than one process calls it at once:

  1. Both callers see DB_ID return null, both issue CREATE DATABASE, and the loser gets SqlException 1801 — "Database 'x' already exists".
  2. The method returned as soon as CREATE DATABASE completed, so the caller's next OpenAsync against the new catalog could still fail — a freshly created SQL Server database briefly refuses logins.

Neither mattered while a single process bootstrapped a single database at startup. Both matter as soon as parallel test workers each provision their own database against a cold container.

Promoting proven code, not writing new logic

SqlServerDatabaseBootstrap in Weasel.EntityFrameworkCore.Tests already solved exactly this and has been stranded where nothing outside those tests could reach it. This moves it into the shipped assembly; that helper is now a thin shim that delegates.

What changed

  • 1801 is caught and treated as success. The IF DB_ID(...) IS NULL CREATE DATABASE form was considered and rejected as the whole answer — SQL Server does not make that pair atomic against a concurrent CREATE, so it only narrows the window. The 1801 catch is what closes it, so the cheap parameterized DB_ID check stays as the fast path and the catch does the real work.
  • The method now polls until the database accepts a connection. Unconditionally, not only when we created it — a concurrent creator leaves us the same window. When it returns, callers can take the postcondition at face value: the database exists and is reachable.
  • The wait is bounded and expires loudly. TimeoutException names the database, points at the knob, and carries the last SqlException as InnerException. Silence after a timeout would be worse than the old behaviour.
  • ] in the database name is escaped. CREATE DATABASE takes no parameters, so the name has to be interpolated into a delimited identifier.

Answering the issue's open questions

Should the retry ceiling be configurable? Yes. DatabaseAvailabilityTimeout defaults to 30s (matching the test helper's 30 × 1s) and DatabaseAvailabilityPollingInterval to 1s. TimeSpan.Zero makes a single attempt and fails fast, which is what a developer with a warm container wants.

Do the other providers need the same treatment? PostgreSQL has the same check-then-create shape and the same race, so PostgresqlMigrator.EnsureDatabaseExistsAsync now catches 42P04 duplicate_database. It needs no availability wait — Postgres accepts connections to a new database as soon as CREATE DATABASE returns, so that half is genuinely SQL-Server-specific. Oracle/MySql/Sqlite are not audited here.

Is EnsureDatabaseExistsAsync the right home? Yes. Provisioning N databases is N calls, and the awkwardness of the DbConnection-shaped signature is a separate concern from the race.

Verification

  • ensure_database_is_safe_under_concurrent_callers races 8 concurrent callers at one new database. Confirmed failing against master (verified by disabling only the 1801 catch) and passing here.
  • times_out_with_a_clear_message_when_the_database_never_accepts_connections uses an offline database as the reproducible stand-in for "created but refusing logins", and pins the message and inner exception.
  • ensure_database_escapes_a_bracket_in_the_database_name covers the identifier escaping.
  • Full Weasel.SqlServer suite green: 343 passed, 8 pre-existing skips.
  • Weasel.EntityFrameworkCore SQL Server tests green (17) running on the delegating bootstrap.

🤖 Generated with Claude Code

…callers (weasel#415)

EnsureDatabaseExistsAsync did a check-then-create against master. Two callers
both see DB_ID return null, both issue CREATE DATABASE, and the loser gets
SqlException 1801, "Database 'x' already exists". The method also returned as
soon as CREATE DATABASE completed, so the caller's next OpenAsync against the
new catalog could still fail -- a freshly created SQL Server database briefly
refuses logins.

Neither mattered while a single process bootstrapped a single database at
startup. Both matter as soon as parallel test workers each provision their own
database against a cold container.

The logic is not new. SqlServerDatabaseBootstrap in the EF Core test project
already solved exactly this and has been stranded there where nothing outside
those tests could reach it. This promotes it into the shipped assembly, and
that helper is now a thin shim that delegates.

* SqlException 1801 is caught and treated as success. The IF DB_ID(...) IS NULL
  CREATE DATABASE form was considered and rejected as the whole answer -- SQL
  Server does not make that pair atomic against a concurrent CREATE, so it only
  narrows the window. The 1801 catch is what actually closes it, so the cheap
  parameterized DB_ID check is kept as the fast path and the catch does the
  real work.

* After the create, the method polls until the database accepts a connection.
  The wait is unconditional rather than only-when-we-created-it, because a
  concurrent creator leaves us the same window. When it returns, callers can
  take the postcondition at face value: the database exists and is reachable.

* The wait is bounded and expires loudly. TimeoutException names the database
  and points at the knob, with the last SqlException as InnerException. Silence
  after a timeout would be worse than the old behaviour.

Answering the issue's open questions:

* The ceiling is configurable. DatabaseAvailabilityTimeout defaults to 30s
  (matching the test helper's 30 x 1s) and DatabaseAvailabilityPollingInterval
  to 1s. TimeSpan.Zero makes a single attempt and fails fast, which is what a
  developer with a warm container wants.

* Other providers: PostgreSQL has the same check-then-create shape and the same
  race, so PostgresqlMigrator.EnsureDatabaseExistsAsync now catches 42P04
  duplicate_database. It needs no availability wait -- Postgres accepts
  connections to a new database as soon as CREATE DATABASE returns, so that half
  is genuinely SQL-Server-specific. Oracle/MySql/Sqlite not audited.

* EnsureDatabaseExistsAsync stays the home for this. Provisioning N databases is
  N calls, and the awkwardness of the DbConnection-shaped signature is a
  separate concern from the race.

Also escapes ']' in the database name, since CREATE DATABASE takes no
parameters and the name has to be interpolated into a delimited identifier.

Regression tests: ensure_database_is_safe_under_concurrent_callers races 8
concurrent callers at one new database and fails against 9.22.0 (verified by
disabling only the 1801 catch); the offline-database test pins the timeout
message and its inner exception. Full Weasel.SqlServer suite green (343 passed,
8 pre-existing skips), Weasel.EntityFrameworkCore SqlServer tests green (17) on
the delegating bootstrap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit f68c69c into master Jul 31, 2026
19 of 20 checks passed
@jeremydmiller
jeremydmiller deleted the fix/415-ensure-database-exists-race branch July 31, 2026 21:27
erdtsieck pushed a commit to erdtsieck/weasel that referenced this pull request Aug 3, 2026
…urrent window

enumerating_while_registrations_land_neither_throws_nor_tears raced the writer
to the finish: it read in a `while (!writer.IsCompleted)` loop and then asserted
`passes > 0`. When the pool scheduled the writer promptly it landed all 5,000
registrations before the first IsCompleted check, so the loop body never ran and
the test asserted nothing about the registry -- green locally, and red in CI on
the trailing `passes > 0` with `Shouldly.ShouldAssertException : passes`
(JasperFx#418, Postgres 15.3-alpine net9.0 job, passed on rerun with no
code change).

The concurrent window is now established by handshake rather than by scheduling
luck. The writer signals after its first registration and the reader blocks on
that before taking any pass; the reader then takes a fixed number of passes and
signals when done; the writer, after registering its new keys, keeps
re-registering them until that signal arrives. It never blocks, so the map is
being actively written to for the whole of the reader's passes, and the vacuous
`passes > 0` assertion is gone.

The invariants the test exists to protect are unchanged: a read never sees fewer
entries than were seeded and never sees a null. Re-registering existing keys does
not move the final count, so `Count == seeded + added` still holds -- and still
fails under the non-atomic read-modify-write setter this test guards against,
where a stale-snapshot write during the churn can drop keys outright.

Verified with 200 consecutive local runs, 0 failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SqlServerMigrator.EnsureDatabaseExistsAsync is not safe under concurrent callers

1 participant